View Issue Details

IDProjectCategoryView StatusLast Update
0001376T99X171.00 SKB EagleSW Issuepublic2021-12-24 09:21
Reporter(ALTech) Younkwang Jung Assigned To(ALTech) Younkwang Jung Due Date
PriorityimmediateSeveritys4-minorReproducibilityhave not tried
Status closedResolutionfixed 
Summary0001376: [Smart3][BTF][VoC] Kernel panic VoC issue.
DescriptionHi Kerwin

kernel panics are occurring in the FXN model.
I asked Amlogic to analyze the kernel panic and updated the results of analysis.

The related jira is as follows.
https://jira.skbroadband.com/browse/BTFAML-790
The logs are on the Jira site.

As a result of analyzing many VoC logs, the following pattern of issues are found.
There are many kernel panics when BT RCU is abnormally disconnected.
=========================================================================================
<4>[226230.027172@1] --> skbrm_raw_event: id=0x80, size=4, data= 80 00 49 00
<4>[226230.027182@1] key down 103
<6>[226230.214723@2] atvr_remove: hdev->name = BMM_BA02_E62 removed, num 1->0 <== BT disconnected
<6>[226230.232847@2] binder: 3471:3737 transaction failed 29201/-1, size 32-0 line 3149
<36>[226230.329963@3] type=1400 audit(1634170192.580:349435): avc: denied { call } for comm="ATVRemoteAudioH" scontext=u:r:hal_audio_amlogic:s0 tcontext=u:r:rc_server:s0 tclass=binder permissive=0
<36>[226230.329981@3] type=1400 audit(1634170192.676:349436): avc: denied { read } for comm="btvpropertyserv" name="u:object_r:shell_prop:s0" dev="tmpfs" ino=1491 scontext=u:r:btvservice_hal:s0 tcontext=u:object_r:shell_prop:s0 tclass=file permissive=0
<36>[226230.880826@0] type=1400 audit(1634170192.676:349436): avc: denied { read } for comm="btvpropertyserv" name="u:object_r:shell_prop:s0" dev="tmpfs" ino=1491 scontext=u:r:btvservice_hal:s0 tcontext=u:object_r:shell_prop:s0 tclass=file permissive=0
<36>[226230.880854@0] type=1400 audit(1634170193.228:349437): avc: denied { read } for comm="HwBinder:3546_1" name="u:object_r:default_prop:s0" dev="tmpfs" ino=1418 scontext=u:r:btvservice_hal:s0 tcontext=u:object_r:default_prop:s0 tclass=file permissive=0
<1>[226231.554415@1] Unable to handle kernel paging request at virtual address 6e656420203a6b
<1>[226231.554423@1] [006e656420203a6b] address between user and kernel address ranges
<0>[226231.554429@1] Internal error: Oops: 96000044 [#1] PREEMPT SMP
<4>[226231.554450@1] Modules linked in: gpio_keypad(O) remote(O) hid_skbrm(O) aml_hardware_dmx(O) mali_kbase(O) amvdec_mavs(O) amlogic_fbc_lib(PO) amvdec_ports(O) vpu(O) encoder(O) amvdec_avs2(O) amvdec_vp9(O) amvdec_vc1(O) amvdec_real(O) amvdec_mmpeg4(O) amvdec_mpeg4(O) amvdec_mmpeg12(O) amvdec_mpeg12(O) amvdec_mmjpeg(O) amvdec_mjpeg(O) amvdec_h265(O) amvdec_h264mvc(O) amvdec_mh264(O) amvdec_h264(O) amvdec_avs(O) stream_input(O) decoder_common(O) firmware(O) media_clock(O) optee_armtz(O) optee(O) snd_soc_tlv320adc3101(O)
<4>[226231.554597@1] CPU: 1 PID: 0 Comm: swapper/1 Tainted: P O 4.9.180 #38
<4>[226231.554605@1] Hardware name: Amlogic (DT)
<4>[226231.554615@1] task: 0000000096b78484 task.stack: 000000002fdca87e
<4>[226231.554634@1] PC is at expire_timers+0x60/0x188
<4>[226231.554644@1] LR is at expire_timers+0x184/0x188
<6>[226231.554654@1] R15 : ffffff800a9febc8, PFN: 29fe
<6>[226231.554662@1] R24 : ffffff800a9feaf8, PFN: 29fe
<6>[226231.554671@1] R28 : ffffff800a9f6000, PFN: 29f6
<6>[226231.554680@1] R30 : ffffff800913019c, PFN: 1130
===================================================================================

[panic scenarios]

1. key down.
2. RC disconnected.
3. panic

1. key down.
2. RC disconnected.
3. auto up
4. key up -1??
5. panic

1. key down abc
2. RC disconnected.
3. auto up
4. key up abc
5. panic

[check point]
on skbrm.ko module,

1. Please check whether there is an exception handling in case the connection is abnormally disconnected.
2. Please check the situation in which the key up comes up twice
   ex>
   <4>[226217.292876@2] key down 103
   <4>[226217.450028@0] key up 103
   <4>[226217.450892@0] key up 103
    
Please check it quickly.
Thank you.
YK.Jung
TagsNo tags attached.
Attach Tags

Users monitoring this issue

User List (ALTech) JunGyu Kim , (ALTech) SY Yoon , (SW) Brent Choi , (SW) Jacky Chiang , (SW) Kerwin Chen

Activities

(ALTech) Younkwang Jung

2021-10-26 11:57

developer   ~0008584

Hi Jim Chen

Please update the progress of this issue.

The kernel panics below are related to this issue, and about 336 times have occurred at user
=======================================================================================================================
Unable to handle kernel paging request pc: PC is at expire_timers+0x60/0x188 lr: LR is at expire_timers+0x184/0x188
Unable to handle kernel paging request pc: PC is at input_handle_event+0x190/0x530 lr: LR is at input_event+0x64/0x88
Unable to handle kernel NULL pointer dereference pc: PC is at input_event+0x34/0x88 lr: LR is at key_up.isra.6+0x50/0x70 [hid_skbrm]
Unable to handle kernel NULL pointer dereference pc: PC is at 0x0 lr: LR is at call_timer_fn+0x3c/0x1d0
Unable to handle kernel NULL pointer dereference pc: PC is at __tcp_retransmit_skb+0x52c/0x6b8 lr: LR is at __tcp_retransmit_skb+0xa4/0x6b8
kernel BUG at common/drivers/android/binder.c:1731! pc: PC is at binder_inc_ref_for_node+0x328/0x330 lr: LR is at binder_inc_ref_for_node+0x17c/0x330
Unable to handle kernel NULL pointer dereference pc: PC is at binder_thread_read+0x490/0x1740 lr: LR is at binder_thread_read+0x484/0x1740
Unable to handle kernel NULL pointer dereference pc: PC is at rb_erase+0x1b0/0x3a8 lr: LR is at __unlink_buffer+0x44/0x78
Unable to handle kernel NULL pointer dereference pc: PC is at rb_erase+0x198/0x3a8 lr: LR is at __unlink_buffer+0x44/0x78
Unable to handle kernel NULL pointer dereference pc: PC is at rb_insert_color+0x10/0x1a0 lr: LR is at __link_buffer+0xbc/0xe8
Unable to handle kernel NULL pointer dereference pc: PC is at _raw_spin_lock+0x24/0x60 lr: LR is at _raw_spin_lock+0x20/0x60
Unable to handle kernel NULL pointer dereference pc: PC is at binder_thread_read+0x2d0/0x1740 lr: LR is at binder_thread_read+0x2c0/0x1740
Unable to handle kernel NULL pointer dereference pc: PC is at tcp_ack+0x614/0x1230 lr: LR is at tcp_ack+0x5e8/0x1230
Unable to handle kernel NULL pointer dereference pc: PC is at expire_timers+0x60/0x188 lr: LR is at expire_timers+0x184/0x188
Unable to handle kernel paging request pc: PC is at timeline_fence_release+0x44/0xa8 lr: LR is at timeline_fence_release+0x3c/0xa8
Unable to handle kernel NULL pointer dereference pc: PC is at __hw_addr_add_ex+0x6c/0x138 lr: LR is at __hw_addr_add_ex+0x88/0x138
Accessing user space memory outside pc: PC is at binder_alloc_new_buf+0x138/0x590 lr: LR is at binder_alloc_new_buf+0x40/0x590
Accessing user space memory outside pc: PC is at binder_delete_free_buffer+0x6c/0x1d0 lr: LR is at binder_free_buf_locked+0x1b8/0x238
kernel BUG at common/kernel/cgroup.c:1259! pc: PC is at cset_cgroup_from_root+0x78/0x98 lr: LR is at cgroup_migrate_add_src.part.10+0x30/0xd8
================================================================================================================================

Please check it quickly
Thank you
YK.Jung

(ALTech) Younkwang Jung

2021-10-27 09:20

developer   ~0008590

Hi Jim Chen

Please update the progress of this issue.

Thank you
YK.Jung

(SW) Jim Chen

2021-10-28 09:21

developer   ~0008611

Hi Mr. Younkwang,

I have add protection to prevent auto-up trigger after atvr_remove.
I'll test it this morning, and will commit it if no side effect found.

(SW) Jim Chen

2021-10-28 16:27

developer   ~0008618

the protection patch is committed to 5.3.2 branch

(ALTech) Younkwang Jung

2021-10-29 07:22

developer   ~0008622

Hi jim

Please explain in detail the points fixed so that SKB can understand them.

Thank you
YK.Jung

(SW) Jim Chen

2021-10-29 09:43

developer   ~0008627

Hi Mr.
button_auto_up is a delayed_work to fire key-up-event when key-up is not received from RCU after 1.5s of key-down.
please check 0000841
and we cancel the delayed_work in atvr_remove.

In brief,
the commit prevent button_auto_up to be called after atvr_remove.

Issue History

Date Modified Username Field Change
2021-10-22 11:33 (ALTech) Younkwang Jung New Issue
2021-10-22 11:33 (ALTech) Younkwang Jung Status new => assigned
2021-10-22 11:33 (ALTech) Younkwang Jung Assigned To => (SW) Kerwin Chen
2021-10-22 11:34 (ALTech) Younkwang Jung Description Updated View Revisions
2021-10-22 11:36 (ALTech) Younkwang Jung Issue Monitored: (ALTech) SY Yoon
2021-10-22 11:36 (ALTech) Younkwang Jung Issue Monitored: (ALTech) JunGyu Kim
2021-10-22 11:36 (ALTech) Younkwang Jung Issue Monitored: (SW) Jacky Chiang
2021-10-22 11:36 (ALTech) Younkwang Jung Issue Monitored: (SW) Brent Choi
2021-10-22 17:21 (SW) Kerwin Chen Assigned To (SW) Kerwin Chen => (SW) Jim Chen
2021-10-26 11:52 (ALTech) Younkwang Jung Issue Monitored: (SW) Kerwin Chen
2021-10-26 11:57 (ALTech) Younkwang Jung Note Added: 0008584
2021-10-27 09:20 (ALTech) Younkwang Jung Note Added: 0008590
2021-10-28 09:21 (SW) Jim Chen Note Added: 0008611
2021-10-28 16:27 (SW) Jim Chen Note Added: 0008618
2021-10-29 07:22 (ALTech) Younkwang Jung Note Added: 0008622
2021-10-29 09:43 (SW) Jim Chen Note Added: 0008627
2021-12-20 18:28 (SW) Kerwin Chen Assigned To (SW) Jim Chen => (ALTech) Younkwang Jung
2021-12-20 18:28 (SW) Kerwin Chen Status assigned => resolved
2021-12-20 18:28 (SW) Kerwin Chen Resolution open => fixed
2021-12-23 16:29 (SW) Jacky Chiang Severity s2-severe => s3-moderate
2021-12-24 09:21 (SW) Kerwin Chen Severity s3-moderate => s4-minor
2021-12-24 09:21 (SW) Kerwin Chen Status resolved => closed